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ABSTRACT 

In the High/Scope Cognitively Oriented Curriculum, 
teachers create an environment in which students develop and apply . 
cognitive capacities in the areas of classification, seriation, 
spatial and temporal relations and in the process of representation 
to the widest ranges of materials and subjects. Learning takes place 
through the child's manipulation and experimentation with objects and 
through his experiences with the cognitive goal areas incorporated 
into his activities by the teacher. 1'he observation procedure used is 
SCOPE (Systematic classroom Observatioi of Pupil Experience) and 
consists of six broad categories: (1) child-adult contacts, (2) 
child-child contacts, (3) child-materi Jl contacts involving reading 
or writing, (4) child-material contacts not involving reading or 
writing, (5) lone, and (6) group size. A separate instrument for 
coding teacher behavior, SCOTE, was also created. Since classroom 
behaviors are directly relevant to curriculum goals, observational 
data can be used for assessing success in meeting those goals. The 
intensive data on individual children nay serve a useful formative 
function since a profile of the interaction patterns of the children 
can be provided to the teacher. (Authorise) 
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The High/Scope Foundation is in its third year of ' - - 
collecting classroom observation data as part of the evalu- 
ation jof our Follow Through model. Over the years, the cur- 
riculum has been in the process of evolving, and our obser- 
vation 'procedures have likewise changed. Part of this change 
has resulted from the natural sorts of modifications that we 
all make as we see results that are less interpretable than 
we had expected, categories that are less reliable than we 
had hoped and definitions that seem inadequate in retrospect. 
But changes have also occurred in our views as to how we can 
best use the observational data. In this paper I would like 
to share some of our experiences with you, try tq explain 
why we view observations the way we do, and give some illus-r 
trative findings from the spring data collection we have just 
completed.. I would also like to present some data we have 
that bears on some of the important methodological issues in 
observational research . 

The High/scope Cognitively Oriented Curriculum 

To set the stage, I should briefly summarize the nature 
of the curriculum that High/Scope sponsors in Follow Through. 
In the High/Scope Cognitively Oriented Curriculum teachers 
create an environment in which children develop and apply 
cognitive capacities in the areas .of Classification, Seria- 
tion, Spatial and Temporal relations and in the process of- 
representation to the widest possible ranges of materials 
and subjects. ! 

To accomplish this, teachers arrange the room by stock* 
ing it with materials that can be actively manipulated to ' 
produce finished products and ongoing processes, with which 
children can identify. These materials typically include 
art and construction materials, games, musical instruments, 
books and magazines, typewriters, old clothes and costumes, 
ingredients for cooking arid experimenting. Children have 
access to the outside. The material is arranged so that 
related material is stored in well defined locations and is 
readily accessible to children. Furniture allows and suggests 
projects and active work by providing work space and storage 
for uncompleted work. 



Teachers establish a schedule for activity that brings 
children into contact with the materials in the room and 
nurtures increasingly complex experiences in which children 
actively manipulate materials to produce products which in- 
clude performances, constructions, art works and. completions 
of self-set tasks . „ The/, schedule provides- for representation 
of experiences through 'planning, preparing and organizing 
before activities are undertaken and representing experi- 
ences after they happen through linguistic and other means 
of representation. Before children begin work tjhey are -} 
asked to dictate; or, write a plan for what they ire going; ' 
to do thaf'Say or for the next several days. Children then 
work to carry out tjieir plans. After a period of work, , 
children represent their experiences by dictating or writi- 
ng stories about what,., they have done and represent what/ 
chey have done tjhxoiigh pictures,,' diagrams,, charades, models 
and verbal descriptions. Teachers plan and carry out small 
group activities! designed to introduce new materials and 



concepts. Large .group meetings ar& scheduled for 
stories arid field trips. 



performances 



Teachers interact with children t}0 exploit the potential 
of materials and activities children have selected. Children 
are encouraged to experience and verbalize concepts and re- 
lationships in the planning or production of processes and 
products. 

The child's role is one of active participation. His 
learning takes place through his manipulation and experimen- 
tation with objects and through: his experiences with the 
cognitive goal areas that teachers incorporate into his 
activities. It is through this active involvement by both 
teacher and child that children's thinking, communication 
and academic skills are strengthened. 

\ ' .- 

When it came to evaluating our program based on this 
curriculum in several field sites, we felt a need to gb 
beyond the traditional outcome measures. For one thing 
these measures did not appear to be sensitive to the changes 
that we felt were takiiig place. Our staff could see things 
happening in the classrooms that somehow were not being, 
reflected in test scores. In ; large part, the goals of the 
curriculum specify a process of development that is seen 
in children's daily behavior. If we could systematically 
record what children were doing, the description of the 
classroom process would, in itself, provide us with evidence 
of the program's effectiveness. 



k% first we viewed observations as\,a way of assessing 
fjow well the. curriculum was being implemented. Our perspec-- — 
/tive has now changed so that the classroom process is viewed, 
in part,, as a program, outcome , or at least., an important facet 
of the total outcome picture. Before discussing this point of 
view more fully, let me back up and ifevidw how we arrived at 
our current position.' ' , J I 



Past Classroom Observation Research 

When High/Scope first began classrpom observation 
research we used systems that had already been developed. 
In our Curriculum Demonstration Project in the mid-sixties, 
the OScAR (Seifert, 1969) , and the PROSE (Sheriff, 1971) 
were used (supplemented by a couple of our own procedures) . 
We -subsequently used jthe "PROSE in a study of program im- 
plementation J. n our Planned Variation Head Start Sites 
(D^loria, Dick, Hanvey and Love, 1972) . The PROSE (devel- 
oped by Medley, Quirk, Sen luck and Ames, 1971, at ETS)is a 
very useful all-purpose observation instrument, using a • 
time-sampling procedure. We got decent reliability with 
it and its categories picked up differences and similarities 
between programs that made sense to us. But we were dis- 
satisfied because it seemed that we should be able to say 
even mor,e„, t about our classrooms. This led us to define 
behavior categories v|for both teacher and child) ,that were 
more specifically expected in classrooms where the Cogni- 
tively Oriented Curriculum was being implemented. We began 
to realize that observation systems cannot be value-free. 
Every definition represents a way of conceptualizing the 
process" one is trying to assess. Even though- all, observatich 
systems might meet the criterion of having categories of 
behavior operationally defined, the selection of behaviors 
to observe reflects (and should reflect, in our view) the 
theoretical biases of those constructing the system. , 

As everyone on this panel knows, it is no easy 'matter 
to define precisely the behaviors you- are interested in and 
to train several observers to cede the behaviors reliably. 
Thus, the first system we developed to our own specifications 
was soon to be revised. Nevertheless, last year we were able 
to learn some very important things about doing classroom 
observations and to demonstrate some of the things others 
have been talking about. 1 




^The findings from the 1972-73 classroom observation study 
are detailed in the Follow Through Progress Report submitted 
by High/Scope to the Office of Education (Morris and Love, 
1973) . 



Methodological findings * One thing we learned was that 
there is no such thing as the index of observer agreement, at 
least for our observation instrument. Last year there were 
three observers and a trainer conducting observations at two 
grade levels at four sites in the fall and in the spring. We 
found differences in observer-trainer agreement across obser- 
vers, within observer across sites, and within sites across 
time. Since our feeling is that unreliable data should not n 
be analyzed, havi r different subsets -of analyzable data* for ^3 
different sites a.id time points greatly complicates the ---"^ 
analysis. 

Another thing we learned about the methodology of class- 
room observation was that the classroom behavior patterns of 
teacher, and children were relatively stable across days. Each 
classroom had been observed for two mornings and two afternoons . 
When the first morning and afternoon were combined and compared 
(by a chi square analysis) with the second morning and afternoon 
the two days looked very similar for six of the seven randomly 
selected classrooms. In two classes there was a significant 
day-to-day difference on only one of the categories; four class- 
rooms differed from day to day on only two to four of the 66 
categories on the instrument. Keep in mind that this stability 
occurred with a procedure in which all children in the classroom 
were observed in a random sequence for shor t per iod s of time. 
We have other data which suggest that when the observer -focuses 
on a small number of children and observes different children 
on different days, there will be greater variation in the "pic- 
ture" of the classroom from day to day. , 1 

Classroom interaction findings . Last year we collected 
preliminary data comparing Follow Through and non-Follow Through 
classrooms in one of our. sites. VJe found several important 
differences between the two sets of classrooms that seemed to 
. support: observation procedures as a way of describing what was, 
happening in the classroom, in the third grade-Follow Through 
classrooms children initiated interactions with adults more 
frequently, received more individual attention from adults, 
were more often drawing or creating picture materials, and were 
more often in small groups. During child-child interactions, , 
Follow Through children were more likely tfyan non-Follow Through 
children to be using rraterials. In interactions with adults, 
the adults in the Follow Through classes were more likely to be 
listening to or watching the children rather than "lecturing, " 
Similar results were obtained. at -the first grade level. These 
findings came from one site where we had control data. During 
1973-74 we have collected Follow Through and non-Follow Through 
data in three different locations. 



Focus on Children 

I 

In the data we have collected this year, we shifted the 
focus of our observations from the classroom to the child. 
One basic reason for this is our general concern with' finding 
improved procedures for assessing the growth and development 
of children as they progress through the curriculum. Observa- 
tion offers a method for assessment that should tell us what 
children are actually doing as they work with materials, .and 
as they interact with their peers and their teachers.' (A 
secondary reason for focusing on children is that I j have al- 
ways found it hard to understand the meaning -of a "classroom" 
behaving with a certain frequency; when the data are a com- 
posite of short time periods from all children in a random 
sequertbb, the un^t of analysis becomes pretty abstract. ) 

to fqcus on individual children, we decided to observe 
each child, for one-half day. This was accomplished by select- 
ing, two 1 children per day and alternating from child to child 
every : ten minutes throughout the day (if the daily routine 
were the s,ame in the morning and afternoon, however, the 
simpler procedure of observing one child in the morning and 
the other in the afternoon v;as followed) . On separate days, 
the teacher was the. focus of the observations. We collected 
two full days of observations on each teacher in this study.. 
Six children were randomly selected from both a second and 
aj third grade classroom from three of our Follow Through 
centers and from three non-Follow Through schools in— the~scimer 
sites. Even with this small sample, data collection took 
four weeks at each site. Since this study was designed to 
be an intensive look at a small but carefully selected sample, 
High/Scope curriculuiri staff selected the "best" second and 
third grade classes in each of these centers. 

The observation procedure is a time sampling category 
system. The observations are made during a two-second in- 
terval, signalled at the beginning and end by a tone recorded 
on a cassette tape and played through an earphone worn by the 
Observer. These two-second observing intervals occurred every 
20 seconds so that approximately 18 seconds between observa- 
tions was available for marking the coding sheet and watching 
the child in preparation for the next observing interval. No 
observations were made during lunch, rest time, library time, 
or during lessons or activities that were not conducted by 
the regular classroom personnel* 



/ 
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The system for observing children is called SCOPE' 
(Systematic Classroom Observation, of Pupil Experience) ,. It 
consists of ( six broad categories, each containing several-; - 
mutually exclusive subcategories or items which define the 
type of event that_is to be coded. The categories 1 can be 
briefly summarized 3s follows: ' V- 

1. Child*-ad ult co ntacts * Items in Category 1 are coded 
when the observed child is paying attention, -to an adult 
during the observation interval. The observer can code 
items indicating that the teacher_and child are engaged 
together in an activity (and IF~sp7 who is talking), 
that there is verbal inter acfci"on"Suv. ho~ activity, or \ 
that the child is passively listening to and/or watching/ 
or that the child is aggressive. 

2. Child-chilci contacts . Whjbn the target child is paying 
attention to another child, the nature of the interaction 
is coded on a cooperative-uncooperative basis (cooperation 
•meaning a joint venture jfchat may require division of labor) 

3. .Child -material contacts 



involving j reading or writing . Six 



items can be coded here,| depending on whether the reading \-\ \ 
or writing is of the child's own story, a peer''s story, or 
of a story from some other source. 



4. Child -material contacts pot involving reading or writing . 
Some materials (sue h as W puzzle) are highly likely to 
'Structure the child's activity? other materials allow 
more creativity or e xploration. The items in th is cate- 
gory code the childTs behavior as manipulating, looking, 

""""f^DT'not using materials ^bhat either structure the activity 
or do not structure it. Additional items. permit coding 
children looking at materials that are not being used, 
fidgeting with materials, or using materials destructively. 

5» Lone . Category 5 was created for coding the child who is 
not interacting with anyone or anything, and was coded 
only if categories 1-4 were not coded* 

6 » Grou p size. The size of the group the \ target child is 
in when an adult is present can be specified in five 
items — sizes of one, two, 3^5, 6-10, and 11+. 



*A coding manual with definitions, examples/ and instructions 
may be obtained from the High/Scope Foundation. 

\ 
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are just beginning/ but 
we are dealingi\w&th the 



Our analyses of th€i data collected in February and March 



I would like to describe the two ways 
observational data. One method is to 



calculate tfte /ijroodifian ' s ((1965) confidence interval for the 
items in each -category. For this analysis the population of _ 
interest is conceptualized as all possible behaviors that a" ~ ~ 
child may emit (see Chasson, 1961) . The time sampling pro- 
cedure thus yields a random sample of the population behaviors. 

The confidence interval concept; is that for some sample 
statistic given some probability' level, say .99/ it is 
possible to define a range of Values for which the probability 
is .99 that the actual population parameter Is contained with- 
in that range. Conversely, the probabilii*y J ls .01 that the 
actual population parameter is, not .bpntained in the interval. 
For these observational data tnis' means that if a child were 
observed for one day ^n^lJw^^ecorded as performing some 
behavior for 20% of those* observations it is possible to 
calculate a range of percentages and make 'probability state- 
ments about that „ range, such as: there is only one chance 
out of 100 that this child's actual percentage of the observed, 
behavior is greater then the upper bound or less than the 
lower bound. When comparing two children on the same be- 
havior it is possible to make probability statements based 
on the pair of confidence intervals calculated for the 
children. If they do not overlap, then the probability 
that the population parameters j are equal is not greater 
than the prbbability that neither of the confidence inter- 
vals contain the population parameter. This joint prob- 
ability is the product of the individual probabilities or 
.99 x .99 which equals .9801. j Therefore when confidence! 
; intervals do not overlap the probability that the popula- 
tion parameters fcjr the pair of children are equal is less 
Ithan 1 - .9801 or jless than .02. 

The, confidence band for the proportion that each item 
is of the t3|al category frequency can be graphed. • Then 
the significant differences between items within a child 
and between <|hildren on a particular item can be determined 
simply by noting which confidence bands do not overlap. 
Figures 1 and 2 illustrate this procedure for two third 
,'^rade children from one of our Follow Through centers. 
Category 1 (Child-adult contact) shows both within-child 
and between-child differences. For Child B, there are two 
significant differences — Item 5 has a higher frequency 
than either Item 1 or 2. For Child A, the only difference 
is that Item 4 (passive listening and watching) occurred 
significantly more often than Item 2 (talking to an adult) . 
Similar -kinds of comparisons can be made between children* 
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It appears that the only difference between the two children . 
on'Category 1 is on Item 5 — Child B spent more time watching 
the observer. On Category 2 (Child-child interactions, the 
two ahildren differ on Items 1 and 2. In addition to obtain- 
ing tljhe reliable differences by examining the confidence bands/ 
we cap look at the actual proportions and make decisions about 
the m^aningfulness. of the differences. The fact that this 
analysis results in-some easily interpiretab.le graphic repre- 
sen'fca-feions suggests that this might also be a nice yay of 
providing feedback to teachers. .... j 

The second way in which we kre treating these data..d}s 
to give each child a "score" which is his frequency) for each 
Of the items. SCOPE would thus be seen as analagous to a 
test with six scales and several ; items per scale. With this 
perspective, the typical inferential statistics can be applied.. 
There' are then, however, tiwo methodological problems that have 
to be dealt with. One is the violation of the assumption of 
independent observations since the children being observed are 
interacting within the same classroom with the same teacher. 
There does not appear to be a solution to this problem, but 
since it results in understating the true differences in any 
t or F tests, I would not be so concerned with this problem. 
The more serious problem is that the dependencies among the 
observational categories result in nohindtpendent tests bf . 
significance if one were -to do, say > an F test for group 

differences on each of the items in the observation system. 

The s imp lest: procedure might be to simply adjust the alpha 
level in all the tests of significance so that "too many" 
significant differences are not found. Other procedures 
such as multivariate analysis of variance or multiple linear 
regression might extract more information from the data, how- 
ever. • |. : -;. 

Although I can't report any findings from these procedures 
yet, I want to mention an interesting finding from our data 
preparations. Since each child is not Observed for an equal 
amount of time, each child has a different total number of 
events coded. To adjust. for this, we took as each child's 
scbre the prdportion that each item frequency was of the 
total number of events. As others have pointed .out, this 
results in highly-skewed, J-shaped distributions for each 
variable, in which the means and standard deviations are 
correlated. V?e found, in fact, that for the 36 SCOPE vari- 
ables the correlation between the means and standard devia- 
tions was .85. The standard recommendation is to apply an 
arcsine transformation to the proportions. However, when 
there are a large number of categories so that the propor- 
tions in all categories are very small, the arcsine transfor- 
mation has practically no effect upon the shape of the dis- 
tributions, and for our data the correlation between tfie 



means and standard deviations of the .transformed data in- 
creased to .89. When we applied a cube root transformation 
many of the distributions became -fairly normal and the cor- 
relation went down to „20. 



Focus On (Teachers 

In addition to observing children/ we created a separate 
observation instrument for . coding teacher behavior , the 
Systematic Classroom Observation of Teacher Experience (SCOTE) 
SCOTE consists of eight categories, eaclv containing several 
items which are mutually exclusive within categories i The I 
items in the eight categories record: 

1. Verbal behavior of the teacher in terms of type of 
question asked/ etc.; 



2. Whether or not. the verbal behavior is related to the 
, material the child may be using; 

3. Whether or not the teacher is responding to a 
child's Verbalization; 

4. The teacher's nonverbal behavior, such as' being 
engaged in an activity with the child or watching % ' 

the child; ' 

■ • * * ■ ■ f 

•' c ' "• 

5. The type of materials the teacher is using when 
she is doing things with a child; , • 

6. Whether or not the teacher interacted with a child 
on a one-to-one basis or 1 as a member of a group; 

' .it 

7. Number of children- in the teacher 's group or the 
number of children she is focusing- on in the in- 

s terest centers; 

8. Whether or not the teacher is interacting with the 
same child or group that she had interacted with 
during the previous observation interval. 



i 



The (teacher observations lend themselves to the same 
kinds of analyses as the child data. In addition, having 
teacher data collected on different days than the child 
data permits correlations between teacher and child vari- . 
ables that are more ref lective of -the true relationship \ 
between teacher and child. If both sets of ol: nervations \ 
were made simultaneously, correlations might be attributed 
to idiosyncratic interaction patterns occurring^ that day. 



Multiple linear regression methods Will be used to explore 
the extent to which the various- teacher variables predict 
child /behavior in :1 the classroom. Our findings v;ill be in- 
cluded in our Folljow Through year-end report to be published 
later this summer. 



Conclusions 

• The analysis of classroom interaction is a complex 
enterprise. We have elected to focus on individual children 
and their teachers. . Since classroom behaviors are directly 
relevant to curriculum goals, observational data can be used 
. for assess.in3_~0.ur success in meeting, those goals. In addi- 
tion, the, intensive data. on individual children may serve a - 
useful formative function since a«profile of the interaction 
patterns of | the children can be provided to. the teacher. 



\ 
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